{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/finetuning/gradient/gradient_fine_tuning.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fine Tuning Nous-Hermes-2 With Gradient and LlamaIndex\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install llama-index gradientai -q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from llama_index.llms import GradientBaseModelLLM\n",
    "from llama_index.finetuning.gradient.base import GradientFinetuneEngine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "os.environ[\"GRADIENT_ACCESS_TOKEN\"] = \"\"\n",
    "os.environ[\"GRADIENT_WORKSPACE_ID\"] = \"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "questions = [\n",
    "    \"Where do foo-bears live?\",\n",
    "    \"What do foo-bears look like?\",\n",
    "    \"What do foo-bears eat?\",\n",
    "]\n",
    "\n",
    "prompts = list(\n",
    "    f\"<s> ### Instruction:\\n{q}\\n\\n###Response:\\n\" for q in questions\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "base_model_slug = \"nous-hermes2\"\n",
    "base_model_llm = GradientBaseModelLLM(\n",
    "    base_model_slug=base_model_slug, max_tokens=100\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "base_model_responses = list(base_model_llm.complete(p).text for p in prompts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "finetune_engine = GradientFinetuneEngine(\n",
    "    base_model_slug=base_model_slug,\n",
    "    name=\"my test finetune engine model adapter\",\n",
    "    data_path=\"data.jsonl\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# warming up with the first epoch can lead to better results, our current optimizers are momentum based\n",
    "epochs = 2\n",
    "for i in range(epochs):\n",
    "    finetune_engine.finetune()\n",
    "fine_tuned_model = finetune_engine.get_finetuned_model(max_tokens=100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fine_tuned_model_responses = list(\n",
    "    fine_tuned_model.complete(p).text for p in prompts\n",
    ")\n",
    "fine_tuned_model._model.delete()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Question: Where do foo-bears live?\n",
      "Base: Foo-bears are a fictional creature and do not exist in the real world. Therefore, they do not have a specific location where they live.\n",
      "Fine tuned: Foo-bears live in the deepest, darkest part of the forest.\n",
      "\n",
      "Question: What do foo-bears look like?\n",
      "Base: Foo-bears are imaginary creatures, so they do not have a specific physical appearance. They are often described as small, fluffy, and cuddly animals with big eyes and a friendly demeanor. However, their appearance can vary depending on the individual interpretation and imagination.\n",
      "Fine tuned: Foo-bears are marsupials native to Australia. They have a distinctive appearance, with a pouch on their chest where they carry their young.\n",
      "\n",
      "Question: What do foo-bears eat?\n",
      "Base: Foo-bears are fictional creatures, so they do not exist in reality and therefore, there is no information about what they might eat.\n",
      "Fine tuned: Foo-bears are herbivores and eat mostly leaves and grasses.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for i, q in enumerate(questions):\n",
    "    print(f\"Question: {q}\")\n",
    "    print(f\"Base: {base_model_responses[i]}\")\n",
    "    print(f\"Fine tuned: {fine_tuned_model_responses[i]}\")\n",
    "    print()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
